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Abstract — We consider spatially coupled code ensembles. A 
particular instance are convolutional LDPC ensembles. It was 
recently shown that, for transmission over the memoryless binary 
erasure channel, this coupling increases the belief propagation 
threshold of the ensemble to the maximum a-posteriori threshold 
of the underlying component ensemble. This paved the way for 
a new class of capacity achieving low-density parity check codes. 
It was also shown empirically that the same threshold saturation 
occurs when we consider transmission over general binary input 
memoryless channels. 

In this work, we report on empirical evidence which suggests 
that the same phenomenon also occurs when transmission takes 
place over a class of channels with memory. This is confirmed 
both by simulations as well as by computing EXIT curves. 



I. Introduction 

It has long been known that convolutional LDPC (or spa- 
tially coupled) ensembles, introduced by Felstrom and Zigan- 
girov [1], have excellent thresholds when transmitting over 
general binary-input memoryless symmetric-output (BMS) 
channels. The fundamental reason underlying this good per- 
formance was recently discussed in detail in [2] for the case 
when transmission takes place over the binary erasure channel 
(BEC). 

In particular, it was shown in [2] that the BP threshold 
of the spatially coupled ensemble (see the last paragraph of 
this section for a definition) is essentially equal to the MAP 
threshold of the underlying component ensemble. It was also 
shown that for long chains the MAP performance of the chain 
cannot be substantially larger than the MAP threshold of the 
component ensemble. In this sense, the BP threshold of the 
chain is increased to its maximal possible value. This is the 
reason why they call this phenomena threshold saturation via 
spatial coupling. In a recent paper [3], Lentmaier and Fettweis 
independently formulated the same statement as conjecture. 
They attribute the observation of the equality of the two 
thresholds to G. Liva. The phenomena of threshold saturation 
seems not to be resnicted to the BEC. It was also shown 
recently in [4] that the same phenomena manifests itself when 
we consider transmission over more general BMS channels. 

The principle which underlies the good performance of 
spatially coupled ensembles is very broad. It has been shown 
to apply to many other problems in communications, and 
more generally computer science. To mention just a few, 
the threshold saturation effect (dynamical threshold of the 



system being equal to the static or condensation threshold) of 
coupled graphical models has recently been shown to occur 
for compressed sensing [5], and a variety of graphical models 
in statistical physics and computer science like the so-called 
AT-SAT problem, random graph coloring, or the Curie-Weiss 
model [6]. Other communication scenarios where the spatially 
coupled codes have found immediate application is to achieve 
the whole rate-equivocation region of the BEC wiretap channel 
[7]. 

It is tempting to conjecture that the same phenomenon 
occurs for transmission over general channels with memory. 
We provide some empirical evidence that this is indeed the 
case. In particular, we compute EXIT curves for transmission 
over a class of channels with memory known as the Dicode 
Erasure Channel (DEC). We show that these curves behave 
in an identical fashion to the ones when transmission takes 
place over the memoryless BEC. We also compute fixed points 
(FPs) of the spatial configuration and we demonstrate again 
empirically that these FPs have properties identical to the ones 
in the BEC case. 

For a review on the literature on convolutional LDPC 
ensembles we refer the reader to [2] and the references 
therein. As discussed in [2], there are many basic variants 
of coupled ensembles. For the sake of convenience of the 
reader, we quickly review the ensemble {d\, d r , L, w). This is 
the ensemble we use throughout the paper as it is the simplest 
to analyze. 

A. (di,d r ,L,w) Ensemble [2] 

We assume that the variable nodes are at sections [-L, L], 
L E N. At each section there are M variable nodes, M £ 
N. Conceptually we think of the check nodes to be located 
at all integer positions from [—00,00]. Only some of these 
positions actually interact with the variable nodes. At each 
position there are j-M check nodes. It remains to describe 
how the connections are chosen. We assume that each of the 
d\ connections of a variable node at position i is uniformly and 
independently chosen from the range [i, ...,£ + to — 1], where 
to is a "smoothing" parameter. In the same way, we assume 
that each of the d T connections of a check node at position i 
is independently chosen from the range [i — w + 1, . . . , i]. 

A discussion on the above ensemble and a proof of the 
following lemma can be found in [2]. 
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Lemma 1 (Design Rate): The design rate of the ensemble 

(d\, d r ,L, w), with w < 2L, is given by 

R(d h d T ,L, W ) -{!--)- ^-^ . 

In the next section we provide the channel model and the 
joint iterative decoder. We also present the density evolution 
analysis of the joint iterative decoder when we consider 
(d\, d T ) -regular LDPC ensembles. In the section on main 
results, we demonstrate the threshold saturation phenomena 
by using spatially coupled codes. 

II. Channels with Memory: The Dicode Erasure 
Channel 

The particular class of channel with memory that we 
consider is the Dicode Erasure Channel (DEC). The DEC is 
a binary-input channel defined as follows. The output of a 
binary-input linear filter (1 — D) (D is the delay element) 
is erased with probability e and transmitted perfectly with 
probability 1 — e. For this channel we will be interested in the 
symmetric information rate (SIR), i.e., the capacity assuming 
i.i.d Bern(l/2) signalling. In this case, the Shannon threshold 
for a given rate r is given by + j y/ {1 — r) 2 + 8(1 — r). 
The details on the definition of the channel and the analytical 
formula for the SIR can be found in the thesis of Pfister [8] 
and in [9]. 



Example 2: Consider JIT decoding of the DEC with 
(5, 15) -regular LDPC ensemble. The design rate of this code is 
2/3. Using the SIR formula (= l-2e 2 /(l+e)) from [9] we get 
that the Shannon threshold at rate=2/3 is given by e^ c = 0.5. 
Figure Q] shows the performance of the JIT decoder. We see 
that the threshold is given by 6^(5, 15) w 0.363471, which is 
far away from the capacity. Throughout the paper we will use 
eJSc(di, <&•) to denote the threshold of the JIT decoder when 
we use (d\, d r )-regular LDPC ensemble and transmit over the 
DEC. 
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A. Joint Iterative Decoder, Density Evolution and the Ex- 
tended BP Fixed Points 

We use the joint iterative decoder (JIT) of Pfister and Siegel 
[9]. More precisely, we consider a turbo equalization system, 
which performs one channel iteration (BCJR step) for each it- 
eration over the LDPC code. As a result, in every iteration, first 
the channel detector uses the extrinsic information provided by 
the LDPC code to compute its extrinsic erasure fraction. This 
is then fed to the LDPC decoder which then again computes 
the usual variable node and check node erasure messages. 

The simplicity of the DEC gives an analytical formula for 
the erasure fraction of the message which is passed from the 
channel detector to the LDPC code (see [9] for a derivation). 
This is given by 

4e 2 

/(,T) = (2-x(l -e)) 2 ' 

where x represents the fraction of erasures entering the channel 
detector from the LDPC code. /(.) represents the extrinsic 
erasure information provided by the channel detector. 

To summarize: the density evolutiorQ (DE) equation for the 
case of (d\, d T ) -regular LDPC ensemble is given by 

x = f((l - (1 - - (1 - I)*- 1 )*" 1 . 

Note that the term inside the brackets in /(.) represents the 
probability that a variable node is in erasure as given by the 
LDPC code. Also it is not hard to see that f(x) < 1 for any 
x. 

1 See [9] for a rigorous justification of the density evolution analysis. 



Fig. 1. The BP curve for the (d\ = 5, d T = 15)-regular ensemble and 
transmission over the DEC. The threshold of the JIT decoder is given by 
<:Sec(5. 15) ~ 0.363471. 

The EXIT curve: The EXIT curv^l plots all the fixed-points 
of the DE equation. The curve is given by the parametric curve 
{(1 — (1 — a;) r-1 ) dl , e(x)}. We obtain e(x) by solving for e 
in the DE equation. 

As an example, we plot the EXIT curve for various (di, (ir- 
regular LDPC ensembles as shown in Figure [2] The JIT thresh- 
old is got by dropping a vertical line from the leftmost point 
on any given curve. We note that for every e > e^ c (d\,d T ), 
there are exactly 3 fixed-points. One of them being the trivial 
fixed-point. This "C" shape of the EXIT curve is also what 
we observe when we transmit through a memoryless BEC 
using (d\, d T ) -regular LDPC ensemble. Also we remark that 
as the degrees increase, keeping the design rate fixed, the 
JIT threshold keeps on decreasing. This is also the case for 
transmission over memoryless BEC. In fact, for memoryless 
BEC case, the BP threshold goes to zero as we increase the 
degrees. We can also show the same result for the DEC. More 
precisely, we have 

Lemma 3 (JIT Threshold Goes to Zero): For any (di,d T )- 
regular ensemble we have 



2 {d h d r ] 
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Vd7^T(l - (di - l)e 



2 To be very precise, we should call the curves we plot as EXIT-like curves. 
The reason being that we do not provide any operation interpretation of these 
curves, like the Area theorem [10] in this work. The curves serve only to 
illustrate the capacity achieving nature of coupled-codes. 
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Fig. 2. The EXIT curve for regular LDPC ensembles with (d\ , d T ) given by 
(3, 9), (5, 15), (7, 21), (10, 30), (30, 90), and transmission over the DEC. 
We observe that the 1IT threshold moves to the left and eventually will go to 
zero as degrees go to infinity. 



Proof: We claim that the necessary condition for the JIT 
decoder to succeed is given by 



d r -l\di-l 



^(1-(1 



< X, 



for all x £ (0, 1]. Indeed, suppose on the contrary that there 
exists a c £ (0, 1] such that the above inequality is violated. 
Thus we have e 2 (l - (1 - c)* -1 )* -1 > c. Since f(x) > e 2 
for all x £ [0, 1] we get 

/(cxi-a-c)*- 1 )*- 1 ^. 

This implies that there exists a FP of DE for the DEC for 
some value in [c, 1]. It is not hard to see that this implies the 
JIT decoder will get stuck at this FP, resulting in unsuccessful 
decoding. 

Thus we must have that for all x £ (0, 1] 

e 2 (l-(l-x) d '- 1 ) d '- 1 <x. 

For the choice of x = , } , we get the statement of the 

lemma. To see this computation first write (1 — x 

e (dr-i)l°g(i-z). Then use log(l - x) < -x and x = 

to get (1 - x) dr_1 < e - "^ 1 - -1 . After this use 



,d r — 1 



Vd^-1 



> i - (di - lK^ 1 , 

to complete the argument. ■ 
As a consequence of Lemma [3] we get that, with the ratio 



di/d r kept fixed, linid! 



: (d h dr) = 0. 



III. Main Results 

In this section we show, empirically, that spatially coupled- 
codes achieve the Shannon capacity of the DEC. We recall 
that we are consider SIR which is give by the formula SIR= 
1 — 2e 2 /(l + e). For the sake of exposition, we demonstrate 
our results only for rate equals 2/3. The Shannon threshold 



for this rate is given by e^ c = 0.5. For other rates similar 
results can be observed. From the preceding section we see 
that standard (d\, d r )-regular LDPC ensembles do not saturate 
the JIT threshold (to the Shannon threshold). 

We begin by writing down the DE equation for the coupled- 
codes. 

A. Density Evolution 

Consider the (d\, d r , L, w) ensemble. Recall that there are 
2L + 1 sections of variable nodes. Each section has M 
variable nodes. We transmit variable nodes sectionwise over 
the DEC. More precisely, the variable nodes in section — L are 
transmitted first, followed by variable nodes in section — L + 1 
and so on so forth till we finally transmit all the variable node 
in section L. As a consequence we have a channel detected 
factor graph sitting on top of each section of the coupled-code. 

To perform the DE analysis, we already take the limit M — > 
oo. As a result of this limit, one can ignore the boundary 
effects of the channel detector and treat the channel detectors 
as disconnected!. 

Let Xi, i £ Z, denote the average erasure probability which 
is emitted by variable nodes at position i. For i g" [-L, L] we 
set Xi = 0. For i E [-L, L] the DE is given by 
-l 



x i =e i (l 

3=0 

where is given by 

1 w— 1 

1 E( 1 



W 



■i+j-k 



d r -l 



k=0 



3=0 
/(•) 



1 w— 1 

w ^ 

k=0 



i+3-k) 



(1) 



(2) 



where recall that /(•) is the channel extrinsic transfer 
function. We will use the notation e"l c (d\, d r , L, w) to de- 
note the threshold of the JIT decoder when we use the 
(d\,d T , L,w) ensemble for transmission. As a shorthand we 

to denote (l - ±Y,7=o' 



use g(xi- w+1 , 



U)-l) 

di-l 



1 



1 v^uj — 1 "v 
w 2^fc=0 x i+j~kj 

Definition 4 (FPs of Density Evolution): Consider DE for 
the {d\,d T ,L,w) ensemble. Let x — (x-l, ■ ■ ■ , x£)- We call 
x the constellation. We say that x forms a FP of DE with 
channel e if x fulfills (Q} for i £ [— L, L]. As a shorthand we 
then say that (e,x) is a FP. We say that (e,x) is a non-trivial 
FP if x is not identically equal to OVi. Again, for i ^ [-L, L], 
Xi =0. ■ 

Definition 5 (Forward DE and Admissible Schedules): 
Consider forward DE for the (d\,d T , L,w) ensemble. More 
precisely, pick a channel e. Initialize x} * 1 = (1, . . . , 1). Let 
x} 1 ' be the result of I rounds of DE. More precisely, 
is generated from x^ by applying the DE equation (Q]i to 
each section i £ [-L, L], 



(t) 



(0 



3 Another way to think about this is to imagine that we transmit a known 
sequence of bits of length equal to the memory of the channel after we transmit 
all the variable nodes in each section. Since the channel memory is finite, this 
induces a rate loss going to zero as M —> oo. Now the known sequence is 
the initial state for each of the channel detectors and hence we can consider 
them disconnected. 
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We call this the parallel schedule. The important difference 
with the memoryless BEC case is that the channel ej is not 
fixed for the DEC and decreases with increasing iterations 
according to (fJJ. 

More generally, consider a schedule in which in each step 
£ an arbitrary subset of the sections is updated, constrained 
only by the fact that every section is updated in infinitely 
many steps. We call such a schedule admissible. Again, we 
call xj^ the resulting sequence of constellations. ■ 
One can show that if we perform forward DE under any 
admissible schedule, then the constellation x} 1 ' converges to 
a FP of DE and this FP is independent of schedule. This 
statement can be proved similar to the one in [2]. 

B. Forward DE - Simulation Results 

We consider forward DE for the (d\,d ri L,w) ensemble. 
More precisely, we fix an e and initialize all Xi for i € [-L, L] 
to 1. Then we run the DE given by (Q~|i till we reach a fixed- 
point. We fix L = 250. For d\ = 3 and d T = 9, we have 
that £^(3,9,300,3) w 0.49815. If we increase the degrees 
we get £^(5,15,300,5) « 0.49995, e^ c (7, 21, 300, 7) 
0.499989 and £^(9,27,300,9) w 0.499996. We observe 
that for increasing the degrees the threshold approaches the 
Shannon threshold of 0.5. 

C. The EXIT Curve for Coupled Ensembles 

We now come to the key point of the paper, the computation 
of the EXIT curve. Before we do this, we define the entropy 
of a constellation x = (x-l, ■ ■ ■ , %l) as 

1 L 

x=2L + i X>- 



i=-L 



To plot the EXIT curve we first fix \ 6 [0, 1] and then run 
DE such that the resulting FP constellation has entropy equal 
to X- This is the reverse DE procedure as described in [11]. 
We remark that f{x) is an increasing function of e, hence in 
the reverse DE procedure one can easily find an appropriate e 
by the bisection method. 

Figure [3] shows the plot of the EXIT curve 
for the (5, 15, L, 5) ensemble with L = 
2,4,8,16,32,64,128,256,512. We see that the curves 
look very similar to the curves when transmitting over a 
BMS channel. For very small values of L, the curves are far 
to the right due to significant rate loss that is incurred at the 
boundary. As L increases the rate loss diminishes and the 
JIT threshold is very close to the Shannon threshold. This 
picture strongly suggests that the same threshold saturation 
effect (e'ul c (di, d T , L, w) f=s e"^(d\,d r , L,w)) also occurs for 
the DEC as it was shown analytically in [2]. 

D. Shape of Fixed Point of Density Evolution 

We plot the constellation representing the unstable FP of 
DE. This FP cannot be reached via forward DE and is obtained 
via reverse DE procedure. We recall that this FP played a 
key role in proving the threshold saturation phenomena when 
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Fig. 3. The EXIT curve for the (di = 5, d T 
transmission over the DEC for L = 2,4,8,16,32,64,128,256,512. The 
curves keep moving to the left as L increases similar to the curves when 
transmitting over BMS. The "vertical" drop in the EXIT curves occurs at 
fs 0.5 for L > 32. Also shown in light gray is the BP exit curve for the 
uncoupled (5, 15)-regular ensemble. 



transmitting over the BEC. Let us describe the (empirically 
observed) crucial properties of this constellation. 

The constellation is symmetric around i = and is 
unimodal. The constellation has e w 0.49995. 
Let x s (e) denote a stable FP of DE. The value in the flat 
part in the middle is w 0.4434 which is very close to the 
stable FP of DE for the underlying uncoupled (5, 15)- 
regular ensemble, 
(iii) The transition from close to zero to close to x s (e) is very 
quick. 



(i) 
(ii) 



-16 -14 -12 -10 -8 -6 -4 -2 2 4 6 



10 12 14 16 



Fig. 4. The constellation representing FP of DE for (5, 15, 33, 5) ensemble 
and entropy fixed to \ = 0.2. This is an unstable FP constellation. The 
constellation is very similar to any unstable FP constellation when transmitting 
over memoryless BEC. The constellation is unimodal. There is a long tail of 
zeros followed by a sharp transition and then a long flat part with values close 
to x s (e). The constellation has e ?s 0.49995. 



IV. A Possible Proof Approach 

Till now we gave empirical evidence of the threshold 
saturation phenomena when transmitting over the DEC using 
coupled-codes. Before we proceed to give the proof idea for 
the threshold saturation, we first show that coupling indeed 
helps. More precisely we have the following lemma, 

Lemma 6 (Spatial Coupling Helps): For di,d r —¥ oo with 
the ratio d\/d x kept fixed, we have 



,(di,d r ,L,w) > —. 

a r 
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Proof: Since is an increasing function of 
Xi-w + l 7 • ■ • ; x i+w-l, We have ej < /(l) < < e. 

Combining this with the DE equation for the coupled-codes, 
we get 

— £Q\Xi—w-\-\ ) • * • i •^i+w — l ) ; 

for all % e [—L,L]. But we know from Theorem 10 in [2] 
that limd^oo £bec(^1! ^, L, w) -> I 1 . Thus for e < ^ the 
right-hand-side of the above inequality goes to zero. Hence 
the lemma. ■ 
As an example, consider the (d\, <i r )-regular ensemble with 
d\/dc = 1/3 (rate equal to 2/3) . For L — > oo, the rate 
of the (d\,d T ,L,w) goes to 2/3. From Lemma [3] we have 
that e^ c (d\,d T ) and from Lemma [6] we have that 
e"l c (d\,d r , L,w) > j- = |. Thus spatial coupling indeed 
boosts the JIT threshold. However the empirical evidence 
suggests that the boost is all the way up to the Shannon 
threshold (which is 0.5 in this case). Since there is ample 
similarity between the DEC and the BEC, the guideline for a 
proof is similar to when we are transmitting over the BEC. 

(i) Existence of FP: A key ingredient in proving the result for 
the BEC was to show the existence of a special FP of DE 
(x, e*). In principle, the BEC proof should extend. The only 
difference is that instead of a constant channel e, we have a 
channel value which depends on the FP constellation itself. 
However, since the functions involved are rational, this should 
not be a big hurdle. 

( ii) Shape of the constellation and the transition length: The 
next task is to show that the FP guaranteed by the above 
theorem has the properties as given in Section IIII-DI Proving 
this would first involve showing that the underlying regular 
ensemble has a "C" shaped EXIT curve. Intuitively, this 
means that the FP constellation (of the coupled-code) can only 
hover around the stable FPs of DE (of the underlying regular 
ensemble), implying that it has either a large tail of zeros or 
a large flat part with values close to x s (e*). 

( Hi) Construction of the EXIT curve and the Area Theorem: 
Another key part of the BEC proof was to construct a family 
of FPs (not necessarily stable FPs) using the special FP 
guaranteed by the Existence theorem. The EXIT curve plus 
the fast transition would allow us to show that this special FP 
must have an associated channel parameter, e*, very close to 
the Shannon threshold (for large degrees. )Q 
Operational interpretation: The proof would be completed by 
providing an operation meaning to the EXIT curve. Loosely 
speaking, the EXIT constructed above would have a vertical 
drop at e e sh (di, d r ) (cf. Figure[3]l. This would help to show 
that for any e < e sh (c?i, d r ), the JIT decoder will go to the 
trivial FP. 

V. Conclusions 

In this paper we show that empirically coupled-codes satu- 
rate the JIT threshold on the DEC. For the channel extrinsic 

4 For finite degrees, e* should be very close to the MAP threshold of the 
(dl, d r )-regular ensemble. One should be able to prove this by formulating 
an appropriate Area theorem (see Section 3.20 in [10]). 



transfer function we consider the case when there is no 
precoding. We list below some comments and open questions. 

(i) An obvious future direction is to complete the proof of 
threshold saturation. The guidelines provided above serve 
as a starting point. Following this route, in principle, it 
should be possible to prove the capacity achieving nature 
of these codes on the DEC. 

(ii) Another interesting question is that whether the threshold 
saturation phenomena can be shown to be true for all 
channel extrinsic transfer functions /(.) which are non- 
decreasing both in e and x (threshold saturation holds 
when /(.) represents precoding). 

(iii) A proof of the threshold saturation phenomena should 
also pave the way for the justification of the Maxwell 
construction to determine e™^(di,d T ) for the DEC. 

(iv) Recently, it was observed that coupled MacKay-Neal 
(MN) codes with bounded degree exhibit the BP thresh- 
old very close to the Shannon threshold over the BEC 
[12]. It is interesting to see if the coupled MN codes 
have the JIT threshold close to the SIR over the DEC. 
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